Remote prompting infrastructure

ABSTRACT

The invention relates to a method and apparatus for sending and receiving prompts to end-users inside and outside the home. A prompt, for example, a message, image, or sound is presented to the end user in order to notify them of a health event, serve as a simple reminder, helps them through their daily activities. The invention includes, for example, the following components: a remote prompting client which runs on the end-user&#39;s home network and is typically associated with a physical display device. This entity has the ability to receive a prompt request from a remote prompting host, and display the prompt to the end-user; and a remote prompting host which runs on the end-user&#39;s home network and has the ability to scan the network and discover all existing remote prompting clients.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.11/529,575 filed Sep. 29, 2006, which is hereby incorporated byreference in its entirety.

FIELD OF INVENTION

The invention relates to a method and apparatus for sending andreceiving prompts to end-users, and in particular, to end-users locatedin a home.

BACKGROUND

Currently, many elder adults rely on a variety of low-tech methods toremember important events such as handwritten messages posted throughoutthe home. Concerned family members must visit/phone frequently to gatherthis type of information which is problematic for the elder adult (theydon't want to be a burden on their family members) as well as the familycaregiver (feelings of guilt for not calling/visiting often enough).

Personal health has been targeted as one of the major growth areas forthe foreseeable future. Elder care is likely to be one of the primary(future) use cases in this area. A remote prompting infrastructureinteroperable with a wide variety of devices that can inform elderadults and remote caregivers of critical/non-critical health events isgoing to be a key building block that will enable elder care relatedsolutions in the home.

Currently there is no advertising tied into the prompting systems outthere. By associating a relevant ad with a specific prompt to eachindividual user, the effectiveness of the ads is increased. Also, sincethe ad is tied to a prompt that the user needs to see, then the ad willbe seen too.

There are ads that are associated with gross classifications liketelevision audience, and there are targeted ads that are associated withfiner classifications like web-browsing choices, but neither of theseare delivered in conjunction with a prompt that the user needs tocomprehend and respond to, nor do they guarantee that the user will seethem.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a high-level platform architecture.

FIG. 2 illustrates an exemplary Transducer Access Point architecture.

FIG. 3 illustrates the inferred services architecture of the system.

FIG. 4 illustrates the device manager architecture of the system.

FIG. 5 illustrates an exemplary display setting for a graphical userinterface.

FIG. 6 illustrates an exemplary history for a graphical user interface.

FIG. 7 illustrates an exemplary remote prompting infrastructure inaccordance with the invention.

FIG. 8 is a flow diagram of pairing an advertisement with a remoteprompting in accordance with the invention.

DETAILED DESCRIPTION

The embodiment of the invention relates to a system for sending promptsto a user in a network including at least one remote prompting client toreceive a prompt and notify the user of an event and a remote promptinghost to scan the network to detect the at least one remote promptingclient, wherein the remote prompting host identifies an event of theuser and sends the prompt to the detected at least one remote promptingclient. Preferably, the remote prompting host is hardware and/orsoftware running on a computer which interfaces with sensors locatedthroughout the network to deter mine the event of the user. Preferably,the remote prompting client is a module on the user's network associatedwith at least one of a network display device, television, phone, cellphone, picture frame display, and mirror. According to a furtherembodiment, the prompt is at least one of a message, image and sound.Preferably, an advertisement is paired with the prompt and/or theadvertisement is loaded to the remote prompting host. According to yetanother embodiment, the remote prompting host determines when theadvertisement is paired and sent with the prompt based on the event,sends the advertisement with the prompt, and notifies a central serverresponsible for the advertisement that the advertisement was sent anddisplayed at the user's network.

According to yet another embodiment, a method is provided for receivinga prompt at a remote prompting client and notifying the user of an eventand scanning the network at a remote prompting host to detect the atleast one remote prompting client, wherein the remote prompting hostidentifies an event of the user and sends the prompt to the detected atleast one remote prompting client. Preferably, the remote prompting hostis hardware and/or software running on a computer which interfaces withsensors located throughout the network to determine the event of theuser. Preferably, the remote prompting client is a module on the user'snetwork associated with at least one of a television, phone, cell phone,picture frame display, and mirror. According to a further embodiment,the prompt is at least one of a message, image and sound and/or anadvertisement is paired with the prompt. In another embodiment, theadvertisement may be loaded to the remote prompting host.

According to this embodiment, the remote prompting host determines whenthe advertisement is paired and sent with the prompt based on the event,sends the advertisement with the prompt, and notifies a central serverresponsible for the advertisement that the advertisement was sent anddisplayed at the user's network.

According to a further embodiment, a system is provided for monitoring auser in a network, the system including a sensor/actuator network todetermine when an event occurs in the system, a computing device tomonitor and prompt the system when an event occurs, the computing deviceincluding an inference engine and a decision manager, a prompting deviceto receive prompts from the computing device and for displaying theprompt to the user, and a gateway or transducer access point to bridgethe sensor/actuator network with the computing device.

Preferably, when the sensor/actuator network determines that an eventhas occurred, sending a response to the computing device. According tothis embodiment, when the computing device receives the response, theinference engine analyzes the response and the decision managerinitiates actuation of the associated prompting device. Preferably, theinference engine collects and interprets the response in order toanalyze data sent from the sensor/actuator, and the decision managerinitiates actuation of the associated prompting device based on rulesdefined by the system.

FIG. 1 illustrates high-level platform architecture. The “left-side” ofthe diagram is characterized by low-power, in some cases proprietaryprotocols, used by the sensors and actuators to handle communications.The “right-side” of the diagram represents the Digital Home, where WiFi,Ethernet, and UPnP are the communication technologies of choice The“Healthy Digital Home” is enabled (in part) by the Transducer AccessPoint (TAP), which bridges the various sensor/actuator networks(specifically Bluetooth and Z-Wave in this case), to WiFi/Ethernet, andexposes the various physical devices “on the left” as UPnP devices “onthe right” such that intelligent control points can perform actions,request state, and subscribe to event notifications.

UPnP is an architecture for pervasive peer-to-peer network connectivityof intelligent appliances, wireless devices, and PCs. It is designed tobring easy-to-use, flexible, standards-based connectivity to ad-hoc orunmanaged networks whether in the home, in a small business, publicspaces, or attached to the Internet. UPnP is a distributed, opennetworking architecture that leverages TCP/IP and the Web technologiesto enable seamless proximity networking in addition to control and datatransfer among networked devices in the home, office, and public spaces.

UPnP is more than just a simple extension of the plug and playperipheral model. It is designed to support zero-configuration,“invisible” networking, and automatic discovery for a breadth of devicecategories from a wide range of vendors. This means a device candynamically join a network, obtain an IP address, convey itscapabilities, and learn about the presence and capabilities of otherdevices.

Protocols supported by the TAP include, for example, Bluetooth andZ-Wave. Obviously, any protocol may be used as would be appreciated bythe skilled artisan. The protocols tend to be flexible, extensiblebuilding blocks that allows radio protocols such as these to communicatewith other devices inside and outside the home.

The following sensors and actuators are examples of devices that may beintegrated into the platform: Scales, Blood Pressure Monitors, PressureSensors, Presence Sensors, Multi-Level Light, Television, PersonalComputers, Bathroom Mirror Displays, Picture Frame Displays, CellPhones, PDAs, etc. Again, this list of devices is exemplary and notexclusive. Due to limitations exhibited by typical wireless sensors(e.g. power profile, transmission range, memory capacity, cost, etc.) aTAP is used to allow communication to take place, as described above.FIG. 2 illustrates an exemplary TAP Architecture. The TAP is intended toserve as a proxy for the low-powered (Bluetooth and Z-Wave) transducers.

An Inference Engine is the computational entity responsible forcomputing and understanding the data collected from the home sensornetwork. Essentially, sensor data is collected, and compared against aprobabilistic model in order to determine the events that the datarepresents. Based on the recognized events, the decision manager mayinitiate local and/or remote actuation. For example; the probabilisticmodel may determine, based on data collected from multiple sensors(e.g., bed pressure sensor, motion sensors, bedroom door contact switch,etc.) that the occupant is in bed.

In FIG. 3, a Sensor Interface is the block responsible for communicatingwith devices exposed by the TAP. This “raw” sensor data is collected andinterpreted by the inference engine. The external interface to theinference engine is referred to as Inferred Services. Inferred Servicesare exposed as devices and are indistinguishable from their physicalcounterparts on the home network. In this way, Inferred Services can bethought of as “virtual sensors” that leverage the processing power of acomputing device to increase the accuracy associated with the data theyare transmitting, as well as collating disparate information fromvarious sensors to represent higher level events. For example: TheActivity Device depicted in FIG. 3 may utilize data from a multitude ofsensors (e.g., motion, light, contact switches, RED, time of day, etc.)to determine that the home occupant is making breakfast. Interestedcontrol points may subscribe to the “making breakfast” event in the samemanner they would use to communicate with a specific physical sensordevice.

A Decision Manager, illustrated in FIG. 4, is an example of a generalcontrol point that subscribes to the Inferred Services. Based on eventnotifications received from the Inferred Services, and rules defined byusers of the system, actuation is initiated either inside or outside thehome environment. The Inferred Services Control Points block is theinterface into the Inferred Services. In this example, both the InferredServices and Decision Manager reside in the same computing device, butthis is not a requirement of the system (nor is having one distinct setof Inferred Services and one distinct Device Manager). Onecharacteristic about the system is the following: If the underlying setof physical sensors change, while the events subscribed to may be more(or less) accurate, there is no need for the Decision Manager to change.

The information retrieved from Inferred Services in conjunction withuser-defined rules will be used within the framework of the platform toconstruct a state machine representing the activities taking place inthe home. The following is an example of the Decision Manager in action:

1. Remote family caregiver accesses rule editor via web interface;

2. RFC creates a rule that specifies the following: If primary occupantis in the bathroom for over 2 hours, send SMS message to my cell phone;

3. Occupant enters bathroom (detected by motion sensor and/or weightscale measurement); and

4. 2 hours elapse (an unusually long time for the home occupant toremain in the bathroom), state machine raises alert to Rule ExecutionEngine, and SMS message is sent via actuator interface alerting theremote caregiver of a possible fall.

For the purposes of this example, the following types of actuation willbe supported by the Decision Manager's Actuator Interface:

-   -   Prompt, including text and color (to designate severity)        overlaid on:    -   Television    -   Picture Frame    -   Bathroom Mirror Display (in addition to ‘text’, the bathroom        mirror will have a more sophisticated display combining the most        recent weight scale measurement and a graphical depiction of        measurement history).    -   Cell Phone (SMS message)    -   (Z-Wave) Light switch on/off (to show communication backchannel        through the TAP).

It is readily understood that the above-mentioned types of actuation isnot limiting, but rather exemplary.

FIGS. 5 and 6 are exemplary user interfaces. FIG. 5, for example,illustrates a display setting for a graphical user interface, while FIG.6 illustrates the history of a graphical user interface.

There exist many reminder or prompting systems on computers orentertainment systems. Some examples are calendar applications thatremind you to go to a meeting or appointment, or a digital videorecorder that prompts you to change the channel to a show that's on.

Future prompting systems may be more intelligent and display prompts forusers on one of many displays in the house based on the nature of theprompt or the location of the person. Such a system would have knowledgeof the nature of the prompt (medical, entertainment, etc.) and thelocation of the person in their home.

This invention, in another exemplary embodiment, may also use the remoteprompting infrastructure to deliver a targeted ad to an individual thatis guaranteed to be seen because it is tied in with the prompt. The adand prompt can be shown continuously until the prompt is acknowledged.Also, since the remote prompting infrastructure will have some knowledgeof the user, advertisers can filter the type of individual they'd liketo target by age, hobbies, or location.

Some Examples:

-   -   A prompt pops up on the TV reminding a person that they need to        refill their prescriptions. At the same time an ad is displayed        for a local pharmacy at which the prescription can be filled.    -   A graph is displayed on the bathroom mirror of a person's weight        and blood pressure. Shortly afterwards the mirror displays an ad        for a local health food store.    -   A reminder comes up for a person to go to a doctor's appointment        to see how their medications are working. At the same time an ad        is displayed for a competing medication with different side        effects or perhaps a generic brand that is cheaper. This way the        person has the new medication fresh in their mind for the        appointment.

FIG. 7 illustrates an exemplary remote prompting infrastructure. Thisinfrastructure, in use with the above described system, discloses amethod and apparatus for sending and receiving prompts to end-usersinside and outside the home where prompt is defined as; a message,image, or sound (or some combination of the three) presented to the enduser in order to; notify them of a health event, serve as a simplereminder, help them through their daily activities (e.g. an elder adultwith mild cognitive decline). The invention includes the followingcomponents:

Remote Prompting Client: This is a software and/or hardware module thatis running on the end-user's home network and are typically associatedwith a physical display device (e.g. television, cell phone, pictureframe display, bathroom mirror with integrated TV, etc.). This entityhas the ability to receive a prompt request from a Remote PromptingHost, and display (either visually, audibly or by touch) the prompt tothe end-user.

Remote Prompting Host: This is a software and/or hardware module that isrunning on the end-user's home network and has the ability to scan thenetwork and discover all existing Remote Prompting Clients. The RemotePrompting Host performs two critical functions:

-   -   Interface to sensor/medical device network to determine where        the end-user is and what they are doing    -   Send prompt to appropriate Remote Prompting Client to notify the        local/remote user of critical or non-critical health events,        reminders, Activities of Daily Living (ADL) assistance, etc.

Example:

-   -   Agnes is an elder adult with type II diabetes.    -   Agnes hasn't measured her weight or blood pressure in several        days.    -   Because her weight scale and blood pressure monitor are both        connected to her personal health system, the system is aware of        this.    -   Agnes is sitting on her couch watching TV.    -   The system uses a pressure sensor in the couch and power clamp        on the television to recognize this activity.    -   The Remote Prompting Host via its interface to the sensor        network knows that Agnes has not measured her weight or blood        pressure recently and that she is watching TV.    -   The Remote Prompting Host sends a prompt to the Remote Prompting        Client associated with the television and a message/icon is        overlaid on top of the TV signal informing Agnes that she needs        to measure her weight and blood pressure, for example using a        network laptop connected to the TV, or an IPTV or living room        PC.    -   Agnes presses a button on her remote control to clear the        prompt; the system is now aware that she has seen it.    -   Agnes measures her weight and blood pressure during the next        commercial (otherwise, she could/would be prompted again at some        point in the future).    -   Agnes has lost a significant amount of weight since her last        weight measurement.    -   The Remote Prompting Host sends a prompt to the Remote Prompting        Client associated with Agnes' daughter's cell phone. An SMS        message is sent to her daughter's cell phone (via an Internet        based. SMS gateway service) informing her daughter of this        potentially dangerous health event.

The remote prompting is part of a larger platform (i.e. architecture)that is responsible for, in addition to the remote prompting, all otheraspects associated therewith.

FIG. 8 is a flow diagram of pairing an advertisement with a remoteprompting in accordance with the invention. This invention would ensurethat advertisements are hitting a targeted and relevant market bypairing the ad with a prompt that a person receives through a remoteprompting infrastructure. Since the prompt would necessarily be specificto the person (take your medication, etc), an appropriate ad to thatsame person would have a much higher likelihood of success. It alsoguarantees that the ad is seen by the target individual. This methodalso can protect the privacy of the people receiving the ads from theadvertisers.

This invention also increases the attractiveness of remote promptinginfrastructures to service providers by creating another revenue stream.

It is readily understood by the skilled artisan that the embodimentsdisclosed herein are merely exemplary and are not intended to limit thescope of the invention.

1-15. (canceled)
 16. A system, comprising: a sensor device configuredto: measure motion data for a user of the sensor device; detect a healthevent of the user based on the motion data; and communicate the detectedhealth event with a low-power wireless protocol; a host deviceconfigured to: receive, from the sensor device using the low-powerwireless protocol, a message for the detected health event; initiate anemergency alert based on the message for the detected health went; send,to a first destination, a first alert communication for the emergencyalert, wherein the first alert communication includes informationassociated with the detected health event and location informationassociated with the user; and send, to a second destination, a secondalert communication for the emergency alert, wherein the second alertcommunication includes the information associated with the detectedhealth event and the location information associated with the user. 17.The system of claim 16, wherein the detected health event indicates apossible fall of the user of the sensor device.
 18. The system of claim16, wherein the first alert communication and the second alertcommunication are automatically sent by the host device in response toinitiation of the emergency alert.
 19. The system of claim 18, whereinthe initiation of the emergency alert occurs after a predetermined timehas elapsed.
 20. The system of claim 16, wherein the second alertcommunication occurs after completion of the first alert communication.21. The system of claim 16, wherein the emergency alert is cancellablein response to input from the user.
 22. The system of claim 16, whereinthe first alert communication comprises an audio message.
 23. The systemof claim 16, wherein the first destination is an entity located remoteto a location of the user.
 24. The system of claim 16, wherein thesecond destination is a phone associated with another user.
 25. Thesystem of claim 24, wherein the second alert communication comprises anSMS message.
 26. An apparatus, comprising: communication circuitry tocommunicate with a sensor device using a low-power wireless protocol; atleast one processor; and memory including instructions that, whenexecuted by the at least one processor, cause the at least one processorto: obtain, from the sensor device using the low-power wirelessprotocol, a message for a detected health event, wherein the sensordevice provides the message based on: measurement of motion data for auser of the sensor device, and detection of the health event of the userbased on the motion data; initiate an emergency alert based on themessage for the detected health event; send, to a first destination, afirst alert communication for the emergency alert, wherein the firstalert communication includes information associated with the detectedhealth event and location information associated with the user; andsend, to a second destination, a second alert communication for theemergency alert, wherein the second alert communication includes theinformation associated with the detected health event and the locationinformation associated with the user.
 27. The apparatus of claim 26,wherein the detected health event indicates a possible fall of the userof the sensor device.
 28. The apparatus of claim 26, wherein the firstalert communication and the second alert communication are automaticallysent in response to initiation of the emergency alert.
 29. The apparatusof claim 28, wherein the initiation of the emergency alert occurs aftera predetermined time has elapsed.
 30. The apparatus of claim 26, whereinthe second alert communication occurs after completion of the firstalert communication.
 31. The apparatus of claim 26, wherein theemergency alert is cancellable in response to input from the user. 32.The apparatus of claim 26, wherein the first alert communicationcomprises an audio message.
 33. The apparatus of claim 26, wherein thefirst destination is an entity located remote to a location of the user.34. The apparatus of claim 26, wherein the second destination is a phoneassociated with another user.
 35. The apparatus of claim 34, wherein thesecond alert communication comprises an SMS message.
 36. At least onenon-transitory machine-readable storage medium comprising instructionsstored thereupon, which when executed by processing circuitry of amobile computing device, cause the processing circuitry to performoperations comprising: obtaining, from a sensor device using a low-powerwireless protocol, a message for a detected health event, wherein thesensor device communicates the message to the mobile computing devicebased on: measurement of motion data for a user of the sensor device,and detection of the health event of the user based on the motion data;initiate an emergency alert based on the message for the detected healthevent; send, to a first destination, a first alert communication for theemergency alert, wherein the first alert communication includesinformation associated with the detected health event and locationinformation associated with the user; and send, to a second destination,a second alert communication for the emergency alert, wherein the secondalert communication includes the information associated with thedetected health event and the location information associated with theuser.
 37. The machine-readable storage medium of claim 36, wherein thedetected health event indicates a possible fall of the user of thesensor device.
 38. The machine-readable storage medium of claim 36,wherein the first alert communication and the second alert communicationare automatically sent in response to initiation of the emergency alert.39. The machine-readable storage medium of claim 38, wherein theinitiation of the emergency alert occurs after a predetermined time haselapsed.
 40. The machine-readable storage medium of claim 36, whereinthe second alert communication occurs after completion of the firstalert communication.
 41. The machine-readable storage medium of claim36, wherein the emergency alert is cancellable in response to input fromthe user.
 42. The machine-readable storage medium of claim 36, whereinthe first alert communication comprises an audio message.
 43. Themachine-readable storage medium of claim 36, wherein the firstdestination is an entity located remote to a location of the user. 44.The machine-readable storage medium of claim 36, wherein the seconddestination is a phone associated with another user.
 45. Themachine-readable storage medium of claim 44, wherein the second alertcommunication comprises an SMS message.
 46. A system, comprising: asensor device, comprising: means for measuring motion data for a user ofthe sensor device; means for detecting a health event of the user basedon the motion data; and means for communicating the detected healthevent; a host device, comprising: means for receiving, from the sensordevice, a message for the detected health event; means for initiating anemergency alert based on the message for the detected health event;means for communicating, to a first destination, a first alertcommunication for the emergency alert, wherein the first alertcommunication includes information associated with the detected healthevent and location information associated with the user; and means forcommunicating, to a second destination, a second alert communication forthe emergency alert, wherein the second alert communication includes theinformation associated with the detected health event and the locationinformation associated with the user.